Interactive ( De ) Weathering of an Image using Physical Models ∗

Images of scenes acquired in bad weather have poor contrasts and colors. It is known that the degradation of image quality due to bad weather is exponential in the depths of the scene points. Therefore, restoring scene colors and contrasts from a single image of the scene is inherently under-constrained. Recently, it has been shown that multiple images of the same scene taken under different weather conditions or multiple images taken by varying imaging optics can be used to break the ambiguities in deweathering. In this paper, we address the question of deweathering a single image using simple additional information provided interactively by the user. We exploit the physics-based models described in prior work and develop three interactive algorithms to remove weather effects from, and add weather effects to, a single image. We demonstrate effective color and contrast restoration using several images taken under poor weather conditions. Furthermore, we show an example of adding weather effects to images. Our interactive methods for image (de)weathering can serve as easy-touse plug-ins for a variety of image processing software. 1 Need for Interactive Deweathering Images taken under bad weather conditions such as fog, mist, rain and snow suffer from poor contrasts and severely corrupted colors. In bad weather, the radiance from a scene point is significantly altered due to atmospheric scattering. The amount of scattering depends on the distances of scene points from the observer. Therefore, restoring clear day contrasts and colors of a scene from a single image taken in bad weather is inherently under-constrained. Recently, there has been considerable research in the vision and image processing communities on color and contrast restoration in bad weather. Deweathering an image has been demonstrated when accurate scene depths are known [6, 8] and when precise information about the atmospheric condition is known [1]. In computer vision, algorithms have been developed to compute scene structure and restore scene contrasts and colors automatically without requiring any information about the atmosphere or scene depths. These algorithms break the ambiguities that exist in deweathering by using multiple ∗This work was supported by a DARPA HumanID Contract (N000-14-00-1-0916) and an NSF Award (IIS-99-87979). images of the same scene taken under different weather conditions [3, 4] or multiple images acquired by varying the imaging optics [7]. In this work, we address the question of how to deweather a single image of a scene without using precise weather or depth information. Recall that previous work showed that multiple images of the same scene are necessary to break the ambiguities in deweathering. However, in many cases, it may not be possible to acquire multiple images. For instance, today, there are millions of pictures corrupted by weather, that are taken by amateur and professional photographers, with virtually no information about the depths or the atmosphere tagged to them. Very often, all we may have is a single photograph of a scene that we wish to deweather. In such cases, we will show that using minimal additional input from the user can successfully break the ambiguities in deweathering an image. We begin by reviewing two scattering models [3, 4] that describe the colors and contrasts of a scene under bad weather conditions. Based on these models, we then present three algorithms to interactively deweather a single image. In all these cases, the user provides simple inputs through a visual interface to our physics-based algorithms for restoring contrasts and colors of the scene. The types of input (for instance, approximate direction in which scene depths increase, or a rough depth segmentation or a region of good color fidelity) may vary from scene to scene, but are easy to provide for a human user. We also use similar interactive methods to add physically-based weather effects to images. We show several results that illustrate effective deweathering of both color and gray-scale images captured under harsh weather conditions. Our algorithms do not require precise information about scene structure or atmospheric condition and can thus serve as easy-to-use plug-ins for existing image processing software, such as Adobe Photoshop . We believe that our interactive methods will make (de)weathering widely applicable. 2 Colors and Contrasts in Bad Weather In this section, we review two single scattering models that describe colors and contrasts of scene points in bad weather. These models are used in our interactive methods to deweather, and add weather to images.